perf: MD outline ~15× + full parse ~8× via line scanner - #145
Merged
Conversation
Outline mode now extracts only node refs, tags, and properties: - Fast direct scanner for #tag / [[page]] / ((block)) - Markdown-specific outline parser with peek dispatch - Skip front-matter probing and position tracking on outline path - Lighter heading title lookahead and outline inline pre-checks Also adds bench/time_parse.exe for Logseq-sized Markdown fixtures. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
|
|
Replace Angstrom block choice for MD outline with a dedicated line scanner (headings, properties, lists, quotes, fences, footnotes) while keeping Outline_inline for refs/tags. Fixes parse_md_outline empty-title edge cases via the new path. ~0.026s vs ~0.30s master on 1.2MB fixture. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
cursor
Bot
changed the title
Tried Parseff (OCaml 5.3 effects) for MD Inline: ~4x slower than Angstrom on short Logseq titles (effects/look_ahead overhead), so not adopted. Instead add a pure-OCaml MD Inline fast path (plain/#tag/[[page]]/ ((block))) with Angstrom fallback. Full parse ~0.19s vs ~0.43s master (~2.3x) on the 1.2MB fixture; outline stays ~15x. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
cursor
Bot
changed the title
Route all Markdown (outline and full) through the line scanner. Full mode uses Inline.parse, Src fences with options/pos_meta, latex env, quote continuation, and heading anchors — ~7–9× vs master on logseq_large while keeping outline_only ~15×. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
The generative MD fixture builder omitted the size field after it was added to Type.Heading, which broke dune builds of that test helper. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
tiensonqin
marked this pull request as ready for review
August 30, 2026 00:27
Outline fast scan falls back to Angstrom Nested_link for [[a [[b]]]]. Property values use Property.property_references in both modes so nested refs (and quoted values) match full parse. Add outline drawer tests. Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
cursor
Bot
changed the title
This was referenced Aug 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Optimize logseq/mldoc Markdown parsing for the Logseq hot path (outline-first, then full).
Performance (1.2MB
examples/logseq_large.md, 3-iter avg, same machine)parse_outline_onlyApproach
parse_outline_only: Skip heavy nested inline once block type is known. Outline keeps tags /[[page]]/((block))/ nested page refs; drops emphasis and similar.Md_outline: Line-oriented MD document parser (no Angstrom blockchoicebacktracking). Used for outline and full Markdown.Inline.parse, fencedSrc(language/options/pos_meta), latex env, quote continuation, heading anchors.try_fast_md_inline/Outline_inline.try_fast_scanfor plain/tag/page/block refs; nested[[a [[b]]]]falls back toNested_link.key::/:PROPERTIES:/#+NAME:withProperty.property_references(nested refs + quoted values) in both modes.Tests
dune runtestgreen (markdown, outline markdown, org, zip, export-md, export-opml).Test plan
dune runtestdune exec ./bench/time_parse.exevs master